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This article aims to determine the effect of a virtual chemistry laboratory on uni¬ 
versity student achievement. The article describes a model of a laboratory course 
that includes a virtual component. This virtual component is viewed as a tool of 
student pre-lab autonomous learning. It presents electronic resources designed 
for a virtual laboratory and outlines the methodology of e-resource application. 

To find out how virtual chemistry laboratory affects student scientific literacy, re¬ 
search skills and practices, a pedagogical experiment has been conducted. Student 
achievement was compared in two learning environments: traditional - in-class 
hands-on - learning (control group) and blended learning - online learning com¬ 
bined with in-person learning (experimental group). The effectiveness of inte¬ 
grating an e-lab in the laboratory study was measured by comparing student lab 
reports of the two groups. For that purpose, a set of 10 criteria was developed. 

The experimental and control student groups were also compared in terms of test 
results and student portfolios. The study showed that the adopted approach blend¬ 
ing both virtual and hands-on learning environments has the potential to enhance 
student research skills and practices in analytical chemistry studies. 

Keywords: tertiary education; e-resources; blended learning; student achievement; 
computer simulation 


Introduction 

Information technology (IT) is being increasingly used and integrated in all areas of 
secondary and tertiary education. The use of technology can facilitate more student- 
centred and inquiry-based learning, and allow for a flipped classroom where content 
is covered outside of class and application and/or feedback occurs during class time 
(Bowen 2012; Esson 2016; Fitzgerald and Li 2015; Kubicek 2005). One of the ways in 
which IT has been employed in chemical education is using various technology tools 
for engagement and assessment, for example, Wiki, Moodle, Web 2.0, Blackboard 
(Ballesta-Claver et al. 2011; Biasutti and El-Deghaidy 2014; Farrell and Krause 2014; 
Franklin and Smith 2015; Morton and Uhomoibhi 2011), allowing to organise both 
asynchronous and synchronous modes of interaction. Electronic tools are feasible 
when designing interactive course textbooks (Nilsson et al. 2010), optimising differ¬ 
ent computational procedures (Young 2011), assessing learning outcome achievement 
and providing feedback. 
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Laboratory course is an essential component of the university chemistry study. To 
which extent can technology be used when organising laboratory exercises, investiga¬ 
tion and experiments? Laboratory work is a typical form of experiential learning. The 
role of experience in learning is well known. It is especially important in learning of 
the sciences. Learning models based on experience date back to the ideas of British 
empiricism and John Locke; John Dewey’s philosophy of pragmatism; Jean Piaget’s 
theory of cognitive development; David Kolb’s experiential learning, etc. These mod¬ 
els imply a concrete experience, an active laboratory experimentation, in which the 
learner ‘touches all the bases’ (Kolb, Boyatzis, and Mainemelis 2001, p. 22) and has 
a tactile contact with the object of study. Effective learning is seen when the learner 
progresses through a cycle of four stages: of (1) having a concrete experience followed 
by (2) observation of and reflection on that experience which leads to (3) the forma¬ 
tion of abstract concepts (analysis) and generalisations (conclusions) which are then 
(4) used to test hypothesis in future situations, resulting in new experiences (Kolb, 
Boyatzis, and Mainemelis 2001). 

At the same time, the organisation of chemical laboratory course at the university 
level and the design of laboratory experiments that are able to fulfil students’ ex¬ 
pectations and faculty goals pose some questions. These issues have been extensively 
discussed in publications of the Towns Research Group (Purdue University Depart¬ 
ment of Chemistry) and Bretz Research Group (Miami University). Numerous works 
have been devoted to the study of chemical laboratory course framed by Ausubel’s 
assimilation theory of cognitive learning (Novak 1993, 2010) and meaningful learn¬ 
ing (Bretz 2001). These studies revealed inconsistency between faculty goals, student 
expectations and learning outcomes across three domains of meaningful learning: af¬ 
fective (attitudes and emotions), psychomotor (physical skills) and cognitive (content 
knowledge growth) (Brandriet, Ward and Bretz 2013; Bruck, Towns and Bretz 2010; 
DeKorver and Towns 2016; Galloway and Bretz 2015a, 2015b). 

An alternative learning environment, a virtual laboratory, seems to contribute to 
occurrence of meaningful learning. It is gaining popularity in many ways: there are 
numerous educational applications, computer-assisted physical and chemical simula¬ 
tions, copying natural phenomena and conditions of an experiment (Basher and Isa 
2006; Gonzalez-Gomez et al. 2015; Seeling 2010; Tatli and Ayas 2013). 

The main tangible and intangible benefits of a virtual lab in addition to a physical 
one are the following. 

First, the general view is that simulations and virtual labs are learner-centred and 
inquiry-based, which promotes higher levels of thinking and retention. It also allows 
students to receive immediate feedback and correct their faulty understanding of a 
concept (Smetana and Bell 2012; Stone 2007). However, the fact that physical labs can 
also reflect inquiry-based learning, for example, through applying the Science Writing 
Heuristic (SWH) approach (Burke, Greenbowe, and Hand 2006; Greenbowe et al. 
2007; Rudd et al. 2001), cannot be overlooked. 

Second, virtual laboratories are seen as a low cost solution for laboratory exper¬ 
iments. Experiments that would be too expensive (either cost of instrumentation 
or supplies), complicated or even dangerous to work with can be recreated safely 
in the virtual environment, thus bridging the gaps found in traditional laboratories 
(Achuthan and Murali 2015; Basher and Isa 2006). In addition, experimentation time 
significantly reduces and routine procedures of processing experimental results be¬ 
come less complex. 
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Third, remote labs are used as supplementary tools for complementing in-person 
laboratory education (Diwakar et al. 2016; Mejias Borrero and Andujar Marquez 
2012), including virtual elements which interact with real ones, thus exposing students 
to blended learning. Students can work independently and at their own pace online, 
learn how to use chemicals and instruments and pre-plan an experiment before carry¬ 
ing it out in class (Borras-Linares et al. 2011). 

Another benefit offers more possibilities to simulate and visualise quite a num¬ 
ber of complex scientific concepts. Students increase their knowledge regarding 
unobservable molecular-level phenomena and acquire better conceptual under¬ 
standing (Chiu, Dejaegher, and Chao 2015; Kolloffel and de Jong 2013; Shegog 
et al. 2012). 

Finally, students become more positive towards using computers for learning. 
They find simulation of laboratory assignments motivating and creating a lot of ex¬ 
perience. Simulation supports students in the accomplishment of cognitive tasks and 
enhances their learning processes (Josephsen and Kristensen 2006). 

However, many comparison studies do not find any difference between physi¬ 
cal and virtual experiments in terms of improved academic performance (De Jong, 
Linn, and Zacharia 2013; Jolley et al. 2016). Moreover, the use of e-labs in teaching 
and learning chemistry, analytical chemistry in particular, fails to meet the university 
needs, as most virtual chemical laboratories are created for secondary schools. Virtual 
labs that can be used for university courses are limited in number, mainly for teaching 
the so-called general chemistry (inorganic and organic). 

Considering this limited application, e-labs cannot replace traditional laboratories 
but can respond to the existing challenges and optimise the learning process. Students 
can benefit from virtual laboratories when ‘learning about the real world’, as they ac¬ 
quire conceptual knowledge and develop science process skills (Finkelstein et al. 2005; 
Jaakkola, Nurmi, and Veermans 2011; Lampi 2013; Peffer et al. 2015; Yang and Heh 
2007). Computer models and simulations, then, are treated as auxiliary cognitive tools 
to acquire experimental and analytical skills, and to develop an ability to interpret 
experimental results, especially during pre-lab lectures. 

There is some evidence to suggest that only a balanced combination of phys¬ 
ical and virtual laboratories can enhance science learning and encourage scien¬ 
tific practices (Chiu, Dejaegher, and Chao 2015; De Jong, Linn, and Zacharia 
2013; Lei et al. 2015). Brinson (2015) reviews recent (post-2005) empirical stud¬ 
ies that focus on comparing learning outcome achievement using traditional lab 
(TL; hands-on) and non-traditional lab (NTL; virtual and remote) participants 
as experimental groups. The findings show that 89% of the 50 reviewed studies 
demonstrate that student learning outcome achievement is equal or higher in NTL 
versus TL across all learning outcome categories (knowledge and understanding, 
inquiry skills, practical skills, perception, analytical skills and social and scientific 
communication). 

The principal aim of this study is to determine the effect of a virtual chemistry 
laboratory on student achievement in the course entitled ‘Analytical Chemistry, Phys¬ 
ical and Chemical Analysis’. The objectives of the article are as follows: It presents 
electronic resources designed for a virtual laboratory and outlines the methodology of 
e-resource application. Then the work assesses the effectiveness of virtual chemistry 
laboratory related to enhancing student scientific literacy, research skills and practices 
through a comparison with the traditional in-class format. 
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Methods 

Course description 

‘Analytical Chemistry, Physical and Chemical Analysis’ is a junior level undergrad¬ 
uate analytical chemistry course taught at the Ural State University of Economics 
(USUE) (Ekaterinburg, Russia). The course is required for bachelor-degree programs 
with majors in Commodities Management and Expertise; Food Processing Technol¬ 
ogy and Public Catering. 


Sample 

In total 50 students participated in our study. Participants were randomly assigned to 
two groups: experimental group (« = 25) and control group (n = 25). The experimental 
group was provided with a virtual lab-based inquiry learning environment created by 
the authors where students were able to perform experiments using the virtual chem¬ 
istry laboratory before in-class experimental work. The control group was taught in a 
traditional - with instructor guidance only - format. 

All students volunteered to participate in the experiment and the assessment 
process. They were curious to monitor their progress and receive feedback on their 
achievements. The experiment followed the procedures fully agreed upon with the 
participants including the issue of confidentiality. 


E-learning resources design 

The virtual lab included the use of computer-based learning environment that was 
developed through interdisciplinary project-based learning (IPBL) described in 
Stozhko et al. (2015) where IPBL is defined as activities performed by IT and chemistry 
students who were directly involved in the design of e-learning resources related to the 
simulation of chemical analysis procedures. However, the present article describes the 
application of IPBL in a different context: the students of the experimental groups 
were not participating in designing virtual laboratory exercises; they just used them as 
a learning resource. 


Procedure 

Students were provided with two learning environments: traditional in-class learning 
and blended learning - online learning combined with in-person learning. Traditional 
laboratory sessions are run as a series of prescriptive steps described in the manual 
which students have to follow in order to obtain some expected results. Laboratory 
sessions also include calculations, writing a lab report and final assessment. 

The blended learning environment combines online learning and in-class instruc¬ 
tion. The laboratory work includes a range of experience-based tasks. To accomplish 
these tasks, students have to apply their research skills under the guidance of the 
teacher. It is expected that students reflect upon their initial predictions, draw con¬ 
clusions and make recommendations. Eventually these accomplishments will lead to 
some quantitative results and conclusions, which form new knowledge both for the 
student and the instructor. Following a prescribed layout, students then prepare a final 
report that is one of the assessment instruments. Other assessment tasks (interviews, 

4 Citation: Research in Learning Technology 2017, 25 : 1968 - http://dx.doi.org/10.25304/rlt.v25.1968 

(page number not for citation purpose) 


Research in Learning Technology 


tests), measuring student ability to run real research experiments, are performed ei¬ 
ther by the instructor or computer tested. 

The virtual component is followed by a physical laboratory experiment. Students 
are expected to apply the skills they acquired during virtual experience. They set ob¬ 
jectives, carry out measurements, receive and process the newly obtained results, come 
to conclusions and make recommendations in the same way as they did within the vir¬ 
tual environment. At the end of each in-class laboratory session, students are assessed 
by context-based tests. 


Assessment and testing 

The effectiveness of integrating an e-lab in the laboratory study was measured by 
comparing: 

1. the lab reports of the experimental group students with the lab reports of the 

control group students; 

2. in-class test results which aimed to assess student achievement of learning ob¬ 

jectives (development of such skills as analysis of product quality, data inter¬ 
pretation and analysis, research skills); 

3. student portfolios which were used to evaluate student learning progress and 

academic achievement over time. 

All the data obtained (lab report assessment and test results) were managed with 
R-Studio and GraphPad Prism 7 software packages. 


Experiment model 

The model of the experiment was framed based on Piaget’s theory of cognitive de¬ 
velopment and David Kolb’ experiential learning and meaningful learning (Ausubel, 
Novak, Bretz). It also exploited the concept of a flipped class when the input was pro¬ 
vided before the practical class. This input or preparedness for the laboratory exercise 
was different for the experimental and the control group. 

The experiment aimed to provide answers to the following questions: 

1. Is there a difference between the resulting applied knowledge and skills in the 

experimental group and in the control group? 

2. Are there any differences in the content of students’ laboratory reports in the 

experimental group and the control group? 

The experiment has three stages: ascertaining, formative and summative. At the ascer¬ 
taining stage, the two groups were put to a pre-lab test to compare students in terms 
of their relevant prior knowledge of chemistry and experimental (laboratory) experi¬ 
ence. Then, all students performed traditional laboratory work and wrote lab reports. 
These results were important for comparing the two groups. 

At the formative stage, the laboratory course was run in the traditional learning 
environment for the control group and in the ’blended’ mode for the experimen¬ 
tal group. Before each practical laboratory class, both experimental and control 
students were given equal time-on-task to get prepared for an in-class lab; thus, 
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both groups exercised a ‘flipped classroom’, but with different content. The con¬ 
trol group students studied the lab manuals; made notes on the main stages of 
the work and completed the tests from the manual. The experimental group was 
working with a virtual laboratory. On the one hand, the virtual element in the 
‘blended’ mode is similar to students’ primary experience, providing the setting for 
a real experiment in the course of experiential learning. On the other hand, it is an 
intermediate step, when students’ expectations and the affective, psychomotor and 
cognitive learning goals may coincide. When the students of both groups came to 
the class, the lab instructor checked their preparedness for a physical experiment 
by interviewing students. 

At the summative stage, both the control and experimental groups performed 
the same physical investigations. They presented their findings in laboratory reports 
which then were analysed by experts (instructors). Finally, context-based testing was 
used to assess both students groups in terms of analytical chemistry content knowl¬ 
edge and experiment skills. 


Results and discussion 

Laboratory course 

The model of a virtual laboratory component of the experimental session is presented 
in Figure 1. 

A virtual laboratory constitutes a component of a laboratory course for the ex¬ 
perimental group students. By an e-lab, we understand software-based application 
designed for modelling real chemical processes, altering parameters and conditions 
under which a laboratory experiment is run. The virtual lab environment presents 
computer images of chemical reagents, glassware and instrumentation, which makes 
the virtual lab highly realistic. Students can work with this application independently 
in the out-of-class setting (e.g. at home) and try to accomplish a research task and 
come to some conclusions. 

The virtual stage of the laboratory session aims to attain the same goals and ob¬ 
jectives (research, practical and pedagogical) as the laboratory in-class work. How¬ 
ever, the virtual component is different as it includes assignments integrated within 
the virtual lab environment, which were designed to structure student experimenta¬ 
tion processes. The virtual component assists students in collecting, systematizing and 
interpreting empirical data. 

The e-lab coursework is structured in such a way that each step is followed by the 
aim of a laboratory task, detailed instructions on how the task should be completed, 
what chemical formulas should be used, how graphs should be plotted and how find¬ 
ings should be interpreted. These comments foster students’ understanding of the 
task, acquisition of empirical experience and data interpretation. 

During in-class laboratory sessions, the experiment group students used real in¬ 
struments and real samples (e. g. toothpaste, juice), applying their virtual experience. 
Conclusions and recommendations formed an essential element of the experimental 
group students’ final reports. The control group students also wrote lab reports as part 
of the assessment process. They used the same layout of the report as the experimen¬ 
tal group. 

The virtual lab is designed on the basis of the computer-assisted learning (CAL) 
system (Stozhko, Tchernysheva, and Mironova 2014). It integrates software modules 
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Figure 1. The model of a virtual component in the laboratory study. 


designed for a range of purposes. Currently, there are over 30 modules, and their 
number is growing. Fourteen modules relate to the virtual laboratory. To illustrate 
how e-lab modules work, we will use three modules. The choice of these modules is 
attributed to at least two reasons. First, these modules deal with the products that 
are mass-consumed (toothpaste, candy, juice). Second, the methods of analysis used 
for these particular products are likely to be the most frequent methods students will 
encounter in their professional life. 
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Thus, the virtual component includes the following units: 

• theoretical background (manuals with the description of analytical methods); 

• preparation (calculation and weighing of samples, preparation of solutions and 
fixing of the exact concentration of titrant); 

• instrumentation (work with virtual electronic scales and other instruments); 

• operation (collection of empirical data); 

• tabulation (construction of tables to show empirical data collected from virtual 
experiments); 

• graphics (plotting on graphs); 

• precision (measurement of precision); 

• control (final test). 

Animated images of instruments are developed with the use of detailed photos 
and video projections of real chemical instrumentation. Newsticker (or scrolling text) 
is used to control the sequence of operations during virtual titration. It displays dig¬ 
ital and textual data explaining the values of the corresponding variables, for exam¬ 
ple, resistance (R) and volume (V) of the model object. Virtual tables show how the 
sample changes over time. Students use a ‘mouse’ to manipulate animated elements 
of the virtual lab. 


Module 1. Using potentiometry for measuring mass fraction of fluoride ion in 
toothpastes 

Students study the basic principles of potentiometry, familiarise themselves with the 
equipment and learn how to collect and process experimental data. Students study 
physical and chemical properties of toothpastes; learn what impact fluoride ions have 
on toothpaste features; and consider the problems associated with these features. Stu¬ 
dents get acquainted with a range of methods that can be used for chemical analysis 
of toothpastes; then compare these methods and justify their choice of the method; 
and formulate the purpose and objectives of the work. (Overall research, practical, 
and pedagogical objectives are stated in the manual). 

Guided by the manual, students learn how to use laboratory instrumentation (elec¬ 
tronic balance, pH meter-millivoltmeter) virtually and follow the procedure of the 
analysis. From a set of virtual samples, students choose an object of study, and con¬ 
duct an experiment that simulates real chemical analysis. Then, they collect and pro¬ 
cess the data, and plot a calibrating curve (by using Microsoft Office Excel). Students 
analyse the obtained results and reach a conclusion. Finally, students write a report 
that should incorporate all the information. The virtual stage is completed with the 
test which enables to assess whether the student is able to conduct a real experiment. 
If the student has passed the test, he/she can now move on to a ‘hands-on’ experiment. 


Module 2. Using pliotoelectrocolorimetry for measuring sugar content in candies 
The experiment kit of a conductivity (photometric) plant includes a portable digital 
LCR-meter (to measure inductance, capacity and resistance), a photoelectrocolorim¬ 
eter, a conductivity cell and a magnetic stirrer. The procedure is similar to the proce¬ 
dure in Module 1. 

8 Citation: Research in Learning Technology 2017, 25 : 1968 - http://dx.doi.org/10.25304/rlt.v25.1968 

(page number not for citation purpose) 


Research in Learning Technology 


Module 3. Using potentiometric titration for measuring acidity in drinks 

The module includes virtual digital scales, a potentiometric device (pH-meter- 
millivoltmeter), a potentiometric cell, a magnetic stirrer and a titrator. The procedure 
is as follows: the calculation and weighing of the primary standard sample (amber 
acid); preparation of virtual solutions (sodium hydroxide, amber acid); use of pre¬ 
pared solutions in virtual potentiometric titration; collecting empirical data during 
virtual titration of amber acid solutions and sample drinks (e.g. juices) with sodium 
hydroxide solution; and computer-based processing of the results of both virtual and 
physical experiments. 


Results of the ascertaining stage 

At the ascertaining stage, the students of the control and experimental groups had a 
pre-test aiming to measure students' prior knowledge of general chemistry. The pre¬ 
test had 25 multiple-choice questions. Students were required to choose two correct 
answers out of four to six alternatives, which, in our opinion, significantly decreased 
the chance of pure guesswork. Each correct answer added one point. The total max¬ 
imum score was 50 points. All test results were analysed using a Shapiro-Wilk's W s 
test (a test of normality), an Ftest of equality of variances and a Student’s t test for 
means comparison. The results are shown in Table 1. 

Based on the results shown in Table 1, the obtained values for W s and p s > 0.05 
allow acceptance of the null hypothesis that there is normal distribution of grades in 
each group. The F test is less than the critical value, which means dispersion equiv¬ 
alence. The below critical values of the / test and p t > 0.05 allow us to conclude that 
in terms of students’ prior knowledge of general chemistry the groups showed no 
difference (a stronger statement about establishing equivalency of the groups cannot 
be made due to limitation of statistical power (Lewis and Lewis 2005). 

At the ascertaining stage, both experimental and control students were com¬ 
pleting the same physical laboratory exercises where the virtual component was not 
present. During the practical class, students were informed that they had to work on 


Table 1. Pre-test results of the control and experimental groups at the ascertaining stage. 


Parameter 

Control group 
in = 25, v = 24) 


Experimental group 
(n = 25, v = 24) 

M 

31.92 


30.88 

a 

5.94 


5.90 

m 

2.57 


2.50 

W (W =0.9186) 

0.9725 


0.9517 

Ps 

0.7089 


0.2737 

nF crmcal = i.7i) 


1.01 


tit .. , =2.011) 

v critical. ' 


0.557 


P, 


0.5804 



Notes: Mis the average score; a is the standard deviation; m is the error of the mean; W is the Shapiro- 
Wilk test experimental value; W ; = 0.9186 is W critical value; p s value is calculated probability of the 
occurrence of normal distribution in groups; F is the experimental value of an Ptest; F , = 1.71 is the 
.Ftest critical value from the use of a = 0.05 for the sample size (n l = n 2 = 25); t is the experimental value 
of an t test; f w . = 2.011 is the t test critical value for degrees of freedom in the samples v = 48; p t value is 
calculated probability of the occurrence of group equivalence. 
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lab reports and they were instructed what requirements their reports should meet. 
The requirements for report writing in fact repeated 10 criteria that experts devel¬ 
oped to evaluate the quality of student reports. They are as follows: 

1. scientific validity of goals and objectives; 

2. clearly formulated objectives; 

3. relevant description of methods and instruments; 

4. critical approach, depth and logic of theoretical background; 

5. ability to make measurements; 

6. ability to process obtained results (including graphics); 

7. ability to critically analyse obtained results, to describe abnormal or unex¬ 
pected results; 

8. ability to perform metrology procedures; 

9. validity and clarity of conclusions; and 

10. practicality and clarity of recommendations. 

The criteria are of a qualitative nature. Each criterion was assigned by a group of 
five raters (experts) with a variable on the four-level scale: ‘not expressed’, ‘poorly 
expressed’, ‘fairly expressed’ and ‘well expressed’. Statistical analysis of inter-rater 
agreement found substantial agreement (McHugh 2012). The raters also took into 
consideration student autonomy and creative approach to report writing. To ensure 
greater objectivity of the evaluation, reports of both groups were shuffled and let¬ 
ter-encoded by support staff (non-experts). After the raters (experts) rated the reports, 
the non-experts decoded the reports from both groups and transferred the experts’ 
grades to tables with student names. 

These tables were used for further expert analysis. The experts were to compare 
the reports of the experimental group participants with the reports of the control 
group students, and to measure the effect of instructional methods (virtual vs. tra¬ 
ditional) on learning outcomes, specifically acquisition of research skills and prac¬ 
tices and scientific reasoning. Table 2 summarises the results of the analysis of the 
experts. 

Following the data from Table 2, the null hypothesis (H 0 ) was proposed that there 
is no difference between students’ report writing skills in the experimental and control 
groups. To test H 0 and to ensure statistical validity of comparing students’ reports, 
normality of distribution of experts’ mean ranks in each of 10 criteria was tested. The 
use of the Shapiro-Wilks test failed to give positive results; hence, the Kruskal-Wallis 
test with Dunn’s posttest for multiple comparisons was applied, which enables to limit 
the chance of type I error. The results are presented in Table 3. 

As can be seen from Table 3, for all criteria, p w value is greater than 0.9999, that 
is, statistically significant differences are not observed. On the basis of the Kruskal- 
Wallis test with Dunn’s posttest, H 0 is accepted as true, which means the absence of 
differences between the experimental and the control groups with regard to students’ 
report writing skills. 


Results of the stimulative stage 

The results of comparing student laboratory reports from the experimental and con¬ 
trol groups based on 10 criteria at the summative stage are presented in Table 4. 
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Table 2. Comparison of student laboratory reports from the experimental and control groups 
at the ascertaining stage. 


Criteria 

Experimental group 
(n = 25, v = 24) 


Control group 
(n = 25, v = 24) 



M 

exp 

o 

exp 

m 

exp 

M , 

G . 

cont 

" ? cont 

1. Scientific validity of 
goals and objectives 

2.53 

0.29 

0.12 

2.57 

0.24 

0.11 

2. Clear formulation of 
objectives 

2.72 

0.35 

0.15 

2.84 

0.26 

0.11 

3. Relevant description of 
methods and instruments 

2.91 

0.34 

0.14 

2.91 

0.33 

0.14 

4. Critical approach, depth 
and logical presentation of 
the theoretical part 

2.60 

0.27 

0.11 

2.55 

0.30 

0.12 

5. Ability to make 
measurements 

2.44 

0.29 

0.12 

2.46 

0.35 

0.15 

6. Ability to process 
obtained results (including 
graphics) 

2.43 

0.32 

0.13 

2.42 

0.26 

0.11 

7. Ability to critically 
analyse obtained results, 
to describe abnormal or 
unexpected results 

2.75 

0.35 

0.15 

2.86 

0.28 

0.12 

8. Ability to perform me¬ 
trology procedures 

2.57 

0.20 

0.09 

2.66 

0.24 

0.10 

9. Validity and clarity of 
conclusions 

2.42 

0.24 

0.10 

2.46 

0.24 

0.10 

10. Practicality and clarity 
of recommendations 

2.16 

0.36 

0.15 

2.08 

0.29 

0.12 


Notes: M and M. ont are the experimental and control group means; o cxp and o com are the standard devia¬ 
tions; m exp and m cont are the errors of the mean; n is the sample size; v is degrees of freedom. 


Table 3. Results of the Kruskal-Wallis test with Dunn’s posttest for multiple comparisons with 
a = 0.05 at the ascertaining stage. 


Criteria" 

MR 

exp 

MR , 

cont 

MR diff 

Significance 

Adjusted p w 

1 

237.6 

252.9 

-15.3 

No 

>0.9999 

2 

318.7 

365.7 

-47.0 

No 

>0.9999 

3 

381.6 

376.2 

5.4 

No 

>0.9999 

4 

254.6 

238.7 

15.9 

No 

>0.9999 

5 

213.5 

203.7 

9.8 

No 

>0.9999 

6 

189.3 

191.6 

-2.3 

No 

>0.9999 

7 

317.7 

351.7 

-34.0 

No 

>0.9999 

8 

235.1 

292.7 

-57.6 

No 

>0.9999 

9 

187.1 

205.1 

-18.0 

No 

>0.9999 

10 

109.3 

87.2 

22.1 

No 

>0.9999 


Note: "The descriptions of criteria are given in Table 2; MR cxp , MR oont are the mean ranks in the experimental 
and control group; MR diff is the difference between mean ranks. 
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Table 4. Comparison of student laboratory reports from the experimental and control groups 
at the summative stage. 


Criteria" 

Experimental group (n - 

25, v = 24) 

Control group (n - 25, v = 24) 

M 

exp 

'’exp 

m 

exp 

M , 

cont 

<7 . 

cont 

m , 

cont 

1 

3.59 

0.25 

0.11 

3.15 

0.31 

0.13 

2 

3.74 

0.23 

0.09 

3.63 

0.26 

0.11 

3 

3.58 

0.25 

0.10 

3.21 

0.27 

0.11 

4 

3.69 

0.25 

0.10 

3.58 

0.34 

0.14 

5 

3.59 

0.27 

0.11 

3.34 

0.30 

0.12 

6 

3.61 

0.25 

0.11 

3.45 

0.25 

0.10 

7 

3.56 

0.29 

0.12 

3.49 

0.34 

0.14 

8 

3.57 

0.26 

0.11 

3.30 

0.29 

0.20 

9 

3.60 

0.28 

0.11 

3.30 

0.30 

0. 12 

10 

3.66 

0.25 

0.10 

3.36 

0.29 

0.12 

Notes: "The descriptions of criteria are given in Table 2. 

M and M cont are the experimental and control group means; a and o rpji( are the standard deviations; 
m and 7w cont are the errors of the mean; n is the sample size; v is degrees of freedom. 


Table 5. Results of the Kruskal-Wallis test with Dunn’s posttest for multiple comparisons for 
a = 0.05 at the summative stage. 

Criteria" 

MR 

exp 

MR , 

cont 

MR diff 

Significance 

Adjusted p w 

1 

288.7 

112.5 

176.2 

Yes 

0.0001 

2 

356.0 

305.9 

50.1 

No 

>0.9999 

3 

280.1 

124.5 

155.6 

Yes 

0.0011 

4 

334.4 

288.7 

45.7 

No 

>0.9999 

5 

292.2 

178.6 

113.6 

Yes 

0.0469 

6 

295.2 

222.4 

72.8 

No 

0.7005 

7 

276.0 

249.9 

26.1 

No 

>0.9999 

8 

278.6 

161.0 

117.6 

Yes 

0.0343 

9 

292.2 

163.0 

129.2 

Yes 

0.013 

10 

318.8 

191.3 

127.5 

Yes 

0.0152 


Note: "The descriptions of criteria are given in Table 2; MR cxp , MR com are the mean ranks in the 
experimental and control group: MR diff is the difference between mean ranks. 


As evident from Table 4, the students from the experimental group showed better 
results than the control group students on each criterion. The data related to statisti¬ 
cal validity of these differences based on the Kruskal-Wallis test with Dunn’s posttest 
for multiple comparisons can be found in Table 5. 

It is apparent from Table 5 that the groups show statistically significant differences 
in some parameters. Students in the experimental group have outperformed students 
in the control group in terms of research skills and practices. Statistically significant 
differences were observed for criteria 1, 3, 5, 8, 9 and 10. Thus, the null hypothesis 
that there is no difference in laboratory report writing skills in the experimental and 
control groups was rejected and the alternative hypothesis that the experimental and 
control students are different in terms of report writing skills was proposed. 
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Table 6. The excerpts from laboratory 3 reports of the experimental and control student groups 
by selected criteria. 

Criteria Control group Experimental group 


Scientific validity of goals 
and objectives 


Relevant description 
of methods and 
instruments 


Goal: to study the general 
concept of the method of 
potentiometric titration 
used for determining acidity 
in juices 
Objectives: 

(1) to determine titrate acidity 
in juice samples with different 
storage conditions 

(2) to compare the obtained 
results with maximum 
permissible concentrations 
pH-50 pH-meter-millivoltme- 
ter; glass and silver chloride 
electrodes 


Validity and clarity of 1. The method of potentiomet- 
conclusions ric titration has been studied 

2. Dependence of juice acidity 
on temperature and storage 
dates has been found 


Goal: to study opportunities of 
the potentiometric method using 
titrable acidity of juices 
Objectives: 

(1) to identify the impact of stor¬ 
age conditions on juice acidity 

(2) to measure the quality of 
sampled juices in terms of 
safe-to-consume aspect 


pH-50 pH-meter-millivoltmeter, 
consisting of pH-150 current 
transducer, power supply unit 
and electrode system (glass and 
silver chloride electrodes) 
Measurement inaccuracy: 

- for the electrode system: 

±5 mV 

- for the current transducer: 

±0.02 

- for pH meter: 0.05 

1. It has been found that the 
method of potentiometric titra¬ 
tion allows to measure acidity of 
a variety of juices with the degree 
of precision that meets the exist¬ 
ing standards 

2. A degree of juice acidity 
grows with higher temperature 
within the studied range from 
10° C to 30° C and similar 
storage dates 


“The laboratory experiment title: Potentiometric titration: Determination of acidity in food products. 


These differences are statistically significant with regard to the following specific 
abilities: clearly formulate research goals and objectives; describe methods and instru¬ 
ments; critically analyse obtained results; make valid conclusions; and offer practical 
recommendations. Table 6 provides some excerpts from students’ lab reports, illustrat¬ 
ing the differences found. 

At the summative stage, context-based testing was used as an assessment tool of 
context-based learning. The latter aims to help students gain a better understanding 
of the world of science, appreciate the importance of chemistry in the modern world 
and see how it relates to people’s lives (Bennett and Lubben 2006). 
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Thus, the tests included tasks that aimed to show how students can apply gained 
knowledge in their future professional experience, for example: 

Choose the relevant method to analyse a product. 

Summarise key concepts behind analytical methods. 

Describe specifications of an instrument. 

Describe the procedure and the use of instruments. 

Calculate statistical errors. 

Determine key figures using the graph. 

Summarise conclusion and give recommendation. 

Similar to pre-tests at the ascertaining stage, these posttests consisted of 25 multi¬ 
ple-choice questions with 4-6 options, 2 of which are correct. The total maximum 
score is 50 points. Examples of tasks are given in Table 7. 

The results of post-lab context-based testing are presented in Table 8. The data 
were analysed using Shapiro-Wilk’s IT test, F test and Student’s t test. 

It is apparent from Table 8 that W s and p s > 0.05 allow us to accept the null 
hypothesis about normal distribution of grades in each group. F is less than the crit¬ 
ical value, which means dispersion equivalence. A greater than critical t value and 
p t < 0.05 indicate that the two samples are different in terms of knowledge of analyt¬ 
ical chemistry at the summative stage of the experiment. 

Additional evidence for an advantageous use of an e-lab as an interactive learning 
tool was provided from student portfolios. A student portfolio is a collection of stu¬ 
dent work and includes different forms of educational evidence such as completed tests, 
lab reports and final assessments. As the staff members of the USUE Department of 
Physics and Chemistry have been working on developing e-leaming resources including 
virtual chemical labs for many years, they use student portfolio to monitor how students 
grow, mature and improve over the whole 4-year bachelor-degree program. Our obser¬ 
vation showed that those students who experienced chemistry blended learning (virtual 
environment + traditional environment) had more evidence of academic or scientific 
accomplishment such as research articles and projects, conference presentations, in their 
portfolios (e.g. projects ‘What kind of water do we drink?’ and ‘Eco-analytical monitor¬ 
ing of snow in Ural cities’ presented at 2014 Eurasia Green Contest; projects ‘Life with¬ 
out oxidative stress’, ‘Secrets of tea chemical composition’, ‘New methods of measuring 
the quality of mineral waters’ and Analysing vegetables for nitrate ions with the use of 
analytical software’ presented at the International Contest of Young Researchers’ and 
Students’ Projects (2013,2014,2015 and 2017). This monitoring allows us to make some 
arguments for a positive impact of portfolio on students’ research skills. 

Online resources may incentivise students to study chemistry and other natu¬ 
ral sciences, and professional subject areas, such as metrology and standardisation, 
commodities management and food processing technology. Virtual learning environ¬ 
ment can also contribute to better (compared to their peers who study in a tradi¬ 
tional way) assessment results, and enhanced written and oral communication skills. 
As inquiry-based labs provide students with exposure to authentic science, they might 
get more willingly involved in real research projects that require application of the 
analytical chemistry knowledge and target the biggest issues of today: environmental, 
technological and social (quality of life). 

It is worth noting that exposure to e-resources could improve student facility 
in using virtual workspaces and information and communication technology liter¬ 
acy. This might help students overcome the problems they often encounter when 
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Table 7. Samples of context-based test tasks. 

Task Options 3 


Which two methods should be used to measure 
antioxidant properties of food products (fruits, 
vegetables, teas, juices, wines, etc.) considering 
the ability of antioxidants to enter into a redox 
reaction with ROS? 

To analyse toothpastes for fluoride ions, 0.5 g of 
two samples of two toothpastes were dissolved 
in a small volume of diluted sulphuric acid, and 
heated in a water bath for 1 h. Then the volume 
was increased to 100 mL. The concentration of 
fluoride ions measured with potentiometry was 19 
mmol/L (Sample 1) and 38 mmol/L (Sample 2). 
Choose two statements that conform to the state 
standard (GOST 7983-99) requirements for the 
value of fluoride ions in toothpastes (i.e. range 
from 0.005% to 0.15% of weight per moral mass). 

To identify the type of mineral water (carbon¬ 
ate or hydrocarbonate) potentiometric titration 
against HC1 solution is used with the record of all 
titration steps. How many titration steps are ob¬ 
served in carbonate and hydrocarbonate mineral 
water? Choose two correct answers. 

Which two-electrode system is used to measure 
acidity (pH) in juices with potentiometry? 


Potentiometry was used to determine thermal sta¬ 
bility of the milk sample by Ca 2+ concentration, 
using the following tabulated data: 


Milk quality 

Ca 2+ ni 2 % 

Highly thermally stable 

<9.5 

Average thermally stable 

9.5-10.5 

Thermally unstable 

>10.5 


1. Radiochemical 

2. Potentiometric 

3. Chromatographic 

4. Coulometric 

5. Spectroscopic 

1. Sample 1 meets the standard 

2. Sample 2 meets the standard 

3. Sample 1 does not meet the standard 

4. Sample 2 does not meet the standard 


1. None in hydrocarbonate water 

2. None in carbonate water 

3. One in hydrocarbonate water 

4. One in carbonate water 

5. Two in hydrocarbonate water 

6. Two in carbonate water 

1. Glass 

2. Glassy-carbon 

3. Gold 

4. Platinum 

5. Silver chloride 

1. is justified 

2. is not justified 

3. can 

4. cannot 


The obtained Ca 2+ concentration was 9.4 mg%. 

To validate the obtained results, a small volume 
of alcohol was added to the milk sample. After 
stirring, the absence of white flakes was observed. 
Analyse the received result. Complete the con¬ 
clusion choosing two appropriate words from the 
alternatives: 

Conclusion'. 

This milk sample is highly thermally stable which 

. by the alcohol test that . be 

used for determining thermal stability of milk. 

“Correct answers are italicised. 
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Table 8. Posttest results of the experimental and control groups at the summative stage. 


Parameter 

Control group 


Experimental group 


(n = 25, v = 24) 


(n = 25, v = 24) 

M 

34.8 


39.2 

a 

5.20 


5.02 

m 

2.20 


2.17 

B. 

II 

p 

kO 

oo 

Ok 

0.9576 


0.9510 

Ps 

0.3679 


0.2643 

F 


1.028 


F , =1.71 

critical 




t 


2.642 


t , =2.011 

critical. 




P, 


0.011 



Notes: M is the average score; a is the standard deviation; m is the error of the mean; W is the Shapiro- 
Wilk test experimental value; W critjcal = 0.9186 is W critical value; p value is calculated probability of the 
occurrence of normal distribution in groups; F is the experimental value of an Ftest; F = 1.71 is the 
Ttest critical value from the use of a = 0.05 for the sample size (tij = n 2 = 25); t is the experimental value 
of a t test; = 2.011 is the t test critical value for degrees of freedom in the samples v = 48; p value is 
calculated probability of the occurrence of group equivalence. 


developing strategies of working with virtual resources (Gal et al. 2015; Stozhko, 
Stozhko, and Shilovtsev 2016), and enable them to cross-apply their IT skills in other 
learning and research domains. Individual students, as mentioned above, participated 
in the development of e-resources for chemistry studies as part of IPBL. 

Our initial observation suggests that a virtual laboratory environment may ensure 
more meaningful student learning. It could also provide students with the opportu¬ 
nity to develop their research skills and practices. 


Conclusions 

It is a widely held view that an e-lab is an effective pre-physical laboratory train¬ 
ing tool as it allows students to develop skills required for conducting hands-on 
experiments. There are some cases when the use of an e-lab is preferred or is the 
only possible way of teaching and learning, for example, distance learning and the 
impossibility to conduct a physical chemical experiment (lack of equipment, safety 
concerns, time constraints). In addition, an e-lab is an effective pre-class training 
tool that can be used prior to physical laboratory experiment as it allows students 
to practice necessary skills. These advantages of the virtual workshop have been 
reported and acknowledged. 

Since the main aim of the present work was to determine the effect of a virtual 
chemistry laboratory on student achievement in the analytical chemistry course, 
it presented the use of original software-based resources in the virtual laboratory 
module format. Our research indicates that virtualised resources can be adopted as 
a supplementary and supportive element in tertiary education. We found out that a 
virtual laboratory experiment that is coordinated with practical laboratory assign¬ 
ments may certainly be a valuable teaching and student engagement tool. A virtual 
laboratory might contribute to developing student scientific literacy and an appre¬ 
ciation for chemistry’s place in society. It might also provide hands-on experience 
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with chemical phenomenon and build a fundamental understanding of chemical 
concepts. Combinations of virtual and physical laboratories offer advantages of 
attainting learning objectives. 

However, the authors acknowledge that the present study has certain limita¬ 
tions. First, our experiment was conducted with only one set of control and ex¬ 
perimental students. Second, our sample size is insufficient (due to the number 
of enrolled students, 50 = one academic group) to generalise the study’s findings 
beyond the study groups. To overcome these limitations, future work is planned 
aiming to enlist more participants, to collect more evidence and to make findings 
more representative. 

We also hope that sharing our research findings might trigger further study of the 
impact of virtual laboratories and their use in science education. 

Acknowledgements 

The authors would like to thank the anonymous reviewers of the journal for their 
constructive comments and valuable suggestions on earlier versions of this article. 


References 

Achuthan, K. & Murali, S. S. (2015) ‘A comparative study of educational laboratories from 
cost & learning effectiveness perspective’, Advances in Intelligent Systems and Computing, 
vol. 349, pp. 143-153. 

Ballesta-Claver, J., et ah, (2011) ‘Using web 2.0 tools to support university teaching in chem¬ 
istry: the new way’, 4th International Conference of Education, Research and Innovation 
(ICERI ), Madrid, Spain, pp. 27-32. 

Basher, H. A. & Isa, S. A. (2006) ‘On-campus and online virtual laboratory experiments with 
Lab VIEW’, IEEE SoutheastCon, Memphis, TN, pp. 325-330. 

Bennett, J. & Lubben, F. (2006) ‘Context-based chemistry: the Salters approach’, International 
Journal of Science Education, vol. 28, no. 9, pp. 999-1015. 

Biasutti, M. & El-Deghaidy, H. (2014) ‘Interdisciplinary project-based learning: an online 
wiki experience in teacher education’, Technology, Pedagogy and Education, vol. 24, no. 3, 
pp. 1-17. 

Borras-Linares, I., et ah, (2011) ‘Development of a virtual laboratory experiment for chemistry 
students’, 3rd International Conference on Education and New Learning Technologies (ED- 
ULEARN), Barcelona, Spain, pp. 3134-3139. 

Bowen, J. A. (2012) Teaching Naked: How Moving Technology Out of Your College Classroom 
Will Improve Student Learning, Jossey-Bass, San Francisco, CA. 

Brandriet, A. R., Ward, R. M. & Bretz, S. L. (2013) ‘Modeling meaningful learning in chem¬ 
istry using structural equation modeling’. Chemistry Education Research and Practice, vol. 
14, no. 4, pp. 421^130. 

Bretz, S. L. (2001) ‘Novak’s theory of education: human constructivism and meaningful learn¬ 
ing’, Journal of Chemical Education, vol. 78, no. 8, p. 1107. 

Brinson, J. R. (2015) ‘Learning outcome achievement in non-traditional (virtual and remote) 
versus traditional (hands-on) laboratories: a review of the empirical research’, Computers & 
Education, vol. 87, pp. 218-237. 

Bruck, L. B., Towns, M. & Bretz, S. L. (2010) ‘Faculty perspectives of the undergraduate chem¬ 
istry laboratory: goals and obstacles to success’, Journal of Chemical Education, vol. 87, no. 
12, pp. 1416-1424. 

Burke, K. A., Greenbowe, T. J. & Hand, B. M. (2006) ‘Implementing the Science Writing 
Heuristic in the chemistry laboratory’. Journal of Chemical Education, vol. 83, no. 7, 
p. 1032. 


17 

(page number not for citation purpose) 


Citation: Research in Learning Technology 2017, 25 : 1968 - http://dx.doi.org/10.25304/rlt.v25.1968 


B. Bortnik et al. 


Chiu, J. L., Dejaegher, C. J. & Chao, J. (2015) ‘The effects of augmented virtual science labora¬ 
tories on middle school students’ understanding of gas properties’, Computers and Educa¬ 
tion, vol. 85, pp. 59-73. 

De Jong, T., Linn, M. C. & Zacharia, Z. C. (2013) ‘Physical and virtual laboratories in science 
and engineering education’. Science, vol. 340, no. 6130, pp. 305-308. 

DeKorver, B. K. & Towns, M. H. (2016) ‘General chemistry students’ goals for chemistry labo¬ 
ratory coursework’. Journal of Chemical Education, vol. 92, no. 12, pp. 2031-2037. 

Diwakar, S., et al., (2016) ‘Complementing education via virtual labs: implementation and de¬ 
ployment of remote laboratories and usage analysis in South Indian villages’, International 
Journal of Online Engineering, vol. 12, no. 3, pp. 8-15. 

Esson, J. M. (2016) ‘Flipping general and analytical chemistry at a primarily undergraduate insti¬ 
tution’, in The Flipped Classroom Volume 2: Results from Practice, eds J. Muzyka & C. Luker, 
pp. 107-125. ACS Symposium Series; American Chemical Society: Washington, DC 

Farrell, S. & Krause, S. (2014) ‘A virtual community of practice to support faculty efforts to 
adopt research-based instructional approaches’, 2014 International Conference on Interac¬ 
tive Collaborative Learning, ICL2014, Dubai, United Arab Emirates, pp. 845-848. 

Finkelstein, N. D., et al., (2005) ‘When learning about the real world is better done virtually: 
a study of substituting computer simulations for laboratory equipment’. Physical Review 
Special Topics-Physics Education Research, vol. 1, no. 1, pp. 010103(1-8). 

Fitzgerald, N. & Li, L. (2015) ‘Using presentation software to flip an undergraduate analytical 
chemistry course’, Journal of Chemical Education, vol. 92, no. 9, pp. 1559-1563. 

Franklin, R. & Smith, J. (2015) ‘Practical assessment on the run: iPads as an effective mobile 
and paperless tool in physical education and teaching’. Research in Learning Technology, 
[online] Available at: http://dx.doi.org/10.3402/rlt.v23.27986 

Gal, Y., et al., (2015) ‘Making sense of students actions in an open-ended virtual laboratory 
environment’, Journal of Chemical Education, vol. 92, no. 4, pp. 610-616. 

Galloway, K. R. & Bretz, S. L. (2015a) ‘Measuring meaningful learning in the general chemis¬ 
try and organic chemistry laboratories: a longitudinal study’, Journal of Chemical Educa¬ 
tion, vol. 92, no. 12, pp. 2019-2030. 

Galloway, K. R. & Bretz, S. L. (2015b) ‘Measuring meaningful learning in the undergraduate 
chemistry laboratory: a national, cross-sectional study’, Journal of Chemical Education, vol. 
92, no. 12, pp. 2006-2018. 

Gonzalez-Gomez, D., et al., (2015) ‘A comprehensive application to assist in acid-base titration 
self-learning: an approach for high school and undergraduate students’, Journal of Chemi¬ 
cal Education, vol. 92, no. 5, pp. 855-863. 

Greenbowe, T. J., et al., (2007) ‘Using the Science Writing Heuristic in the general chemistry 
laboratory to improve students’ academic performance’, Journal of Chemical Education, 
vol. 84, no. 8, pp. 1371. 

Jaakkola, T., Nurmi S. & Veermans, K. (2011) ‘A comparison of students’ conceptual under¬ 
standing of electric circuits in simulation only and simulation-laboratory contexts’, Journal 
of Science Education and Technology, vol. 48, no. 1, pp. 71-93. 

Jolley, D. F., et al., (2016) ‘Analytical thinking, analytical action: using prelab video demon¬ 
strations and e-quizzes to improve undergraduate preparedness for analytical chemistry 
practical classes’, Journal of Chemical Education, vol. 93, no. 11, pp. 1855-1862. 

Josephsen, J. & Kristensen, A. K. (2006) ‘Simulation of laboratory assignments to support 
students’ learning of introductory inorganic chemistry’, Chemistry Education Research and 
Practice, vol. 7, no. 4, pp. 266-279. 

Kolb, D. A., Boyatzis, R. E. & Mainemelis, C. (2001) ‘Experiential learning theory: previous research 
and new directions’, Perspectives on Thinking, Learning, and Cognitive Styles, vol. L pp. 227-247. 

Kolloffel, B. & de Jong, T. (2013) ‘Conceptual understanding of electrical circuits in secondary 
vocational engineering education: combining traditional instruction with inquiry learning 
in a virtual lab’. Journal of Engineering Education, vol. 102, no. 3, pp. 375-393. 


18 

(page number not for citation purpose) 


Citation: Research in Learning Technology 2017, 25 : 1968 - http://dx.doi.org/10.25304/rlt.v25.1968 


Research in Learning Technology 


Kubicek, J. P. (2005) ‘Inquiry-based learning, the nature of science, and computer technology: 
new possibilities in science education’, Canadian Journal of Learning and Technology, vol. 
31, no. 1, [online] Available at: http://cjlt.csj.ualberta.ca/index.php/cjlt/article/view/149/142 

Lampi, E. (2013) The Effectiveness of Using Virtual Laboratories to Teach Computer Networking 
Skills in Zambia, Doctoral dissertation, Virginia Polytechnic Institute and State University, 
Virginia, [online] Available at: https://vtechworks.lib.vt.edu/bitstream/handle/10919/22013/ 
Lampi_E_D_2013.pdf ?sequence= 1 

Lei, Z., et al., (2015) ‘A DC motor position control system in a 3D real-time virtual laboratory 
environment based on NCSLab 3D’, International Journal of Online Engineering, vol. 11, 
no. 3, pp. 49-55. 

Lewis, S. E. & Lewis, J. E. (2005) ‘The same or not the same: equivalence as an issue in educa¬ 
tional research’, Journal of Chemical Education, vol. 82, no. 9, pp. 1408-1412. 

McHugh, M. L. (2012) ‘Interrater reliability: the kappa statistic’, Biochemia Medica, vol. 22, 
no. 3, pp. 276-282. 

Mejias Borrero, A. & Andujar Marquez, J. M. (2012) ‘A pilot study of the effectiveness of 
augmented reality to enhance the use of remote labs in electrical engineering education’, 
Journal of Science Education and Technology, vol. 21, no. 5, pp. 540-557. 

Morton, W. & Uhomoibhi, J. (2011) ‘E-laboratory design and implementation for enhanced 
science, technology and engineering education’, Campus-Wide Information Systems, vol. 
28, no. 5, pp. 367-377. 

Nilsson, G., et al., (2010) ‘A web-based spectroscopy tutorial for the inorganic chemistry labo¬ 
ratory’, Journal of Chemical Education, vol. 87, no. 7, pp. 758-759. 

Novak, J. D. (1993) ‘Human constructivism: a unification of psychological and epistemological 
phenomena in meaning making’, International Journal of Personal Construct Psychology, 
vol. 6, no. 2, pp. 167-193. 

Novak, J. D. (2010) ‘Learning, creating, and using knowledge: concept maps as facilitative tools 
in schools and corporations’, Journal of e-Learning and Knowledge Society, vol. 6, no. 3, 
pp. 21-30. 

Peffer, M. E., et al., (2015) ‘Science classroom inquiry (SCI) simulations: a novel method to 
scaffold science learning’, PLoS One, vol. 10, no. 3, [online] Available at: http://journals. 
plos.org/plosone/article?id=10.1371/journal.pone.0120638 

Rudd, J. A., et al., (2001) ‘Using the Science Writing Heuristic to move toward an inquiry-based 
laboratory curriculum: an example from physical equilibrium’, Journal of Chemical Educa¬ 
tion, vol. 78, no. 12, pp.1680-1686. 

Seeling, R (2010) ‘Work in progress: portable student labs implementation’, Proceedings— Fron¬ 
tiers in Education Conference (FIE)- Arlington, VA, pp. F2F1-F2F3. 

Shegog, R., et al., (2012) ‘Virtual transgenics: using a molecular biology simulation to impact 
student academic achievement and attitudes’, Research in Science Education, vol. 42, no. 5, 
pp. 875-890. 

Smetana, L. K. & Bell, R. L. (2012) ‘Computer simulations to support science instruction and 
learning: a critical review of the literature’. International Journal of Science Education, vol. 
34, no. 9, pp. 1337-1370. 

Stone, D. (2007) ‘Teaching chromatography using virtual lab exercises’. Journal of Chemical 
Education, vol. 84, no. 9, pp. 1488-1496. 

Stozhko, D. K., Stozhko, K. P. & Shilovtsev, A. V. (2016) Social Security of the Person in the 
Information Society: Theoretical and Methodological Aspect, [in Russian], Ural State Agri¬ 
cultural University, Ekaterinburg. 

Stozhko, N., et al., (2015) ‘Interdisciplinary project-based learning: technology for improving 
student cognition’, Research in Learning Technology, vol. 2, p. 27577, [online] Available at: 
http ://dx. doi.org/10.3402/rlt. v23.27577 

Stozhko, N., Tchernysheva, A. & Mironova, L. (2014) ‘Computer assisted learning system for 
studying analytical chemistry’, Chemistry, vol. 23, no. 4, pp. 607-613. 


19 

(page number not tor citation purpose) 


Citation: Research in Learning Technology 2017, 25 : 1968 - http://dx.doi.org/10.25304/rlt.v25.1968 


B. Bortnik et al. 


Tatli, Z. & Ayas, A. (2013) ‘Effect of a virtual chemistry laboratory on students’ achievement’, 
Educational Technology & Society, vol. 16, no. 1, pp. 159-170. 

Yang, K. Y. & Heh, J. S. (2007) ‘The impact of internet virtual physics laboratory instruction 
on the achievement in physics, science process skills and computer attitudes of lOth-grade 
students’, Journal of Science Education and Technology, vol. 16, no. 5, pp. 451—461. 

Young, V. Y. (2011) ‘Using Texas instruments emulators as teaching tools in quantitative 
chemical analysis’, Journal of Chemical Education, vol. 88, no. 10, pp. 1442-1443. 


20 

(page number not for citation purpose) 


Citation: Research in Learning Technology 2017, 25 : 1968 - http://dx.doi.org/10.25304/rlt.v25.1968 


